feat(plugin_repo): add trialware check#1319
Conversation
Add new Trialware_Check under plugin_repo category to detect
trialware and locked built-in functionality in plugin submissions.
Scans PHP and JS files for five pattern groups:
- License key gates
- Pro/premium plan gates
- Trial period expiration gates
- Usage quota limits
- Payment/subscription gates
Each match reported as error with dedicated result code
(trialware_{type}_candidate). Patterns designed to minimize
false positives on legitimate external service integrations.
Fixes WordPress#1313
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
|
Tested during WCEU 2026 Contributor Day. Manual testing:
Automated checks:
Result:
Note:
|
|
Thanks for working on this. I reviewed the PR and also used ChatGPT to help compare it against our internal trialware scanner. The conclusion is that this is a good start, but I don’t think it satisfies the issue yet. The main gap is that the issue asks for an AI-assisted candidate workflow, not a direct regex-to-error check. Right now this PR:
The internal scanner this is based on works differently:
That distinction is important because trialware detection has a high false-positive risk. For example, a plugin may mention a premium version, use an external service API key, or include license/update-related wording without actually locking functionality bundled in the WordPress.org plugin. Those should not become final blocking errors. To move this closer to the requested behavior, I think this needs:
So I would not consider this complete yet. It is a useful skeleton for the static candidate detection piece, but it is missing the key AI confirmation/discard workflow from the issue acceptance criteria. |
…re-check # Conflicts: # docs/checks.md # includes/Checker/Default_Check_Repository.php
…ew pass - add trialware_ prefix to AI_Analyzer::get_ai_prompt_map() - add prompts/ai-review-trialware.md defining genuine gate vs false positive - broaden Trialware_Check regex patterns (activation codes, free trial wording, to unlock/limit reached phrasing, lite-version gating) - add false-positive-shaped fixtures (external API key, separate premium plugin mention, EDD-style license) to the clean test plugin Addresses PR feedback from davidperezgar: candidates now flow through the same --ai runner pass already used by EscapeOutput/NonceVerification/Obfuscation/etc, instead of a new bespoke per-check AI workflow. Refs WordPress#1319
|
Thanks for the detailed review. Good catch on the missing AI confirm/discard step. Looked into how plugin-check already handles this for other checks. Turns out there's a generic mechanism already built for exactly this: So instead of building a second, bespoke AI workflow just for trialware, I wired the
This keeps trialware consistent with how every other AI-reviewed check works, and the candidate/confirm/discard behavior you asked for comes from the existing runner pass rather than new plumbing. Full details in the updated PR description. Let me know if you'd rather see dedicated |
Summary
Adds a new
Trialware_Checkunder theplugin_repocategory to detect potential trialware and locked built-in functionality in plugins submitted to WordPress.org.The check scans PHP and JS files for five categories of gating patterns: license key gates, pro/premium plan gates, trial period gates, usage quota limits, and payment/subscription gates.
Fixes #1313
Changes
includes/Checker/Checks/Plugin_Repo/Trialware_Check.phpNew check class extending
Abstract_File_Check. DefinesPATTERN_GROUPSconstant with five pattern groups, each with regex patterns, result code, and description message. Deduplicates per-file matches within each group to avoid noise.Result codes:
trialware_license_gate_candidatetrialware_pro_premium_gate_candidatetrialware_trial_gate_candidatetrialware_quota_gate_candidatetrialware_payment_gate_candidateincludes/Checker/Default_Check_Repository.phpRegistered
trialwareslug mapping toTrialware_Checkinstance.docs/checks.mdAdded
trialwarerow to the available checks table.Response to review (davidperezgar)
Reviewer asked for candidate-only detection with an AI confirm/discard step, matching issue #1313's acceptance criteria, rather than reporting every regex match straight as an error.
Investigation found plugin-check already has this exact mechanism, generically, not per-check:
AI_Analyzer::get_ai_prompt_map()(includes/Traits/AI_Analyzer.php) maps check-code prefixes to a prompt file, andAbstract_Check_Runner::run()runsanalyze_results_with_ai()after all checks finish (opt-in via CLI--aiflag / admin AI toggle), tagging each issueis_false_positive+ explanation. The CLI (Plugin_Check_Command::split_false_positive_results()) and Admin AJAX already split confirmed vs. false-positive for display. Every AI-reviewed check today (EscapeOutput, NonceVerification, DirectDatabaseQuery, Obfuscation, SettingSanitization, PluginUpdater) uses this same generic pass — none of them call AI to decide error vs. discard inside the check class itself.Rather than duplicating that infrastructure with a new bespoke per-check AI workflow, this update wires
trialwareinto the existing generic pass:includes/Traits/AI_Analyzer.php— addedtrialware_prefix toget_ai_prompt_map(), pointing at a new prompt file. All fivetrialware_*_candidatecodes match this prefix.prompts/ai-review-trialware.md(new) — prompt defining genuine locked-functionality gating vs. false positive, explicitly covering the review's examples: external service API keys, a separate premium plugin/product, generic marketing copy, and harmless license/update-checker wording.Trialware_Check.php— broadenedPATTERN_GROUPSper the review's examples: activation codes, "free trial ended/expired" wording, "to unlock ... feature" / "limit reached" phrasing, and lite/free-version gating. Kept the existing five_candidateresult codes. Added a class docblock explaining the AI-candidate relationship so it's clear this check intentionally does not call the AI client itself.This keeps candidates flagged the same way as every other AI-reviewed check, without a second, incompatible AI-review code path.
Testing
Test 1: Plugin with trialware patterns (should flag errors)
Fixture:
test-plugin-trialware-with-errorscontainsis_licensed(),is_pro(),trial_expired(),quota_exceeded(), andhas_paid()calls.Expected: 5 errors reported, one per result code.
Test 2: Plugin without trialware patterns (should pass clean)
Fixture:
test-plugin-trialware-without-errorscontains standard admin page, settings save, textdomain loading, plus the new false-positive-shaped functions added in response to review.Expected: 0 errors, 0 warnings.
Test 3: Quality checks
Note: the branch had fallen behind trunk (
mergeable_state: dirty); mergedtrunkin as part of this update to pick upAI_Analyzerand resolve two trivial append-only conflicts indocs/checks.mdandDefault_Check_Repository.php.AI Usage Disclosure
Claude Code was used to: investigate the existing AI-review architecture (
AI_Analyzer,Abstract_Check_Runner, CLI false-positive splitting) referenced above, merge trunk and resolve conflicts, wire thetrialware_prefix into the prompt map, draftprompts/ai-review-trialware.md, broaden the regex patterns per review feedback, and add the false-positive test fixtures.